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ABSTRACT 


The  concept  of  a  test  bed  is  a  controlled/  artificial,  and 
reproducible  environment  for  comparative  testing  of  ASW 
predictive  mechanisms.  Concepts  for  a  test  bed  are  presented 
in  terms  of  a  computer  program.  Some  of  the  underlying  the¬ 
oretical  problems  are  addressed  such  as  the  role  of  Bayes' 
Theorem,  completeness  of  mission  specification,  and  estimat¬ 
ing  input  probabilities. 


PREFACE 


The  material  in  Chapter  II  is  primarily  the  work  of  P. 
Luster  and  that  in  Chapter  III  primarily  of  L.  Dodson.  Re¬ 
organization  of  parts  of  the  original  manuscript  and  addition 
of  certain  introductory  and  amplifying  comments  were  suggested 
by  the  editorial  staff. 


I .  INTRODUCTION 


This  paper  presents  research  preliminary  to  the  develop¬ 
ment  of  a  test  bed  for  ASW  predictive  mechanisms.  The  concept 
of  a  test  bed  is  that  of  a  controlled/  artificial/  and  repro¬ 
ducible  environment  in  which  various  predictive  schemes  can  be 
tested  and  compared.  Here,  the  controlled  environment  is  con¬ 
ceived  as  a  computer  program  which  can  be  used  to  simulate  a 
variety  of  ASW  situations  to  varying  levels  of  detail. 

Chapter  II  of  this  paper  presents  some  of  the  considerations 
and  theoretical  problems  Underlying  the  test  bed  concept.  That 
chapter  addresses  the  role  of  Bayes'  Theorem  in  the  test  bed, 
the  abstract  structure  of  the  test  bed  and  missions  represented 
as  contingency  trees,  the  problem  of  completeness  of  mission  spec¬ 
ification,  the  test  bed  viewed  as  a  general  sum  game,  and,  finally 
problems  in  estimating  probability  inputs  for  the  test  bed. 

Chapter  III  contains  design  objectives  and  concepts  developed 
for  the  test  bed.  A  notably  new  approach  is  the  generalized  treat 
ment  of  weapon  systems,  platforms,  sensors,  and  communications 
among  ASW  units.  The  paper  closes  with  the  bibliography. 
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This  paper  is  the  third  in  a  series  related  to  prediction 
in  ASW.  The  majority  of  the  material  in  this  paper  is  not  pred¬ 
icated  on  the  concepts  presented  in  the  earlier  two.  However, 
an  understanding  of  the  material  in  the  two  previous  papers  will 
complement  certain  points  made  in  this  paper.  The  two  earlier 
papers  are  entitled  "Contingency  Trees,  Contingency  Plans,  and 
Utility  Functions",  and  "Selected  Research  into  Predictive 
Mechanisms  for  Antisubmarine  Warfare".  A  fourth  paper  entitled 
"Fundamental  Problems  in  Applying  Models  and  Predictive  Mech¬ 
anisms"  is  planned. 
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II.  UNDERLYING  CONSIDERATIONS 


A.  THE  CONCEPT  OF  A  TEST  BED 

A  means  of  testing  complex  predictive  mechanisms  is  fre¬ 
quently  desired,  one  which  is  less  costly  and  less  risky  than 
simply  building  the  mechanisms  and  seeing  how  well  they  do. 
Simulation  within  a  test  bed  is  one  approach  to  this  problem. 

Conceive  of  the  test  bed  as  being  modularized.  One  module 
represents  the  predictive  mechanism,  and  the  other  modules  repre¬ 
sent  the  various  parts  of  reality  that  give  information  to  and  re¬ 
ceive  information  from  the  predictive  mechanism.  Modules  repre¬ 
senting  competing  predictive  mechanisms,  or  competing  versions  of 
the  same  predictive  mechanism,  may  be  alternately  "plugged  into" 
the  test  bed.  In  this  way  performance  of  the  various  predictive 
mechanisms  in  the  same  environments  can  be  compared, ^  and  the  ef¬ 
fects  of  environment  on  the  performance  of  each  of  the  predictive 
mechanisms  can  be  noted.  Statistical  tests  can  be  devised  to  help 
select  the  predictive  mechanisms  best  suited  to  each  type  of 


This  seems  to  be  the  only  practical  method  of  comparing  such 
dissimilar  predictive  mechanisms  as’  modified  Kalman  filter  devices 
and  Bayesian  predictions  in  a  wide  variety, of  complex  environments. 
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environment;  or,  the  results  of  such  statistical  tests  can  be 
pooled  with  other  considerations,  such  as  costs,  speed,  and 
reliability,  to  arrive  at  a  selection. 

The  test  bed  approach  is  particularly  attractive  when  sta¬ 
tistical  characteristics  of  the  operating  environment  are  not, 
perhaps  cannot  be,  well  specified.  For,  the  modularization 
permits  a  wide  range  of  possible 'environments  to  be  tested. 
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THE  ROLE  OF  BAYES'  THEOREM  IN  A  TEST  BED 


Now  Bayes'  Theorem  is  itself  a  predictive  mechanism  -  in 
fact,  it  is  the  predictive  mechanism  par  excellence  whenever  the 
information  necessary  for  its  application  is  available.  Why? 
Because  under  these  conditions  the  correctness  of  the  results  of 
any  other  predictive  mechanism  are  to  be  judged  by  comparison  with 
the  results  of  applying  Bayes'  Theorem.  They  are  completely  cor¬ 
rect  only  if  they  agree  completely  with  the  results  of  Bayes ' 
Theorem,  and  are  approximately  correct  only  to  the  extent  that 
they  approximate  those  results.  All  this  fellows  from  the  obser¬ 
vation  that  Bayes'  Theorem  is  in  fact  a  theorem,  in  the  rigorous 
mathematical  meaning  of  the  term  "theorem". 

The  main  problem  with  Bayes '  Theorem  is  that  the  information 
required  for  applying  it  is  sometimes  not  available.  For  example, 
some  general  sum  games  provide  examples  of  situations  where  there 
is  no  general  agreement  among  experts  as  to  what  constitutes 
"optimal"  a  priori  probabilities  (of  the  opponent  playing  each  of 
his  possible  pure  strategies) ,  and  where  observed  relative  fre¬ 
quencies  vary  widely  among  individuals.  When  a  priori  probabili¬ 
ties  are  unknown,  Bayes'  Theorem  still  determines  what  the  a 
posteriori  probabilities  would  be  as  a  function  of  what  the  a 
priori  probabilities  might  be;  and  knowledge  of  this  functional 
relationship  may  itself  be  useful.  But,  however  ignorant  model- 
makers  may  be  about  the  statistical  characteristics  of  the  real-  • 
world  environment  within  which  predictive  mechanisms  are  to  ulti¬ 
mately  function,  they  can  be  well  informed  of  the  statistical 
characteristics  of  the  simulated  environment  built  into  a  test  bed. 
The  statistical  characteristics  of  the  simulated  environment  de¬ 
termine  values  of  the  a  priori  probabilities  -  in  fact,  of  all  the 
probabilities  -  required  for  the  application  of  Bayes'  Theorem. 


.  ■  2 
Applied  within  a  simulated  environment,  and  applied  consistently 

with  the  statistical  laws  governing  ithe  simulation,  Bayes’ 

Theorem  is  invincible!  ’  ; 

■  ,  i 

»  » 

This  is  not  to  say  that  predictions  or  decisions  made  by 

th'e  .correct -use  of  Bayes’  Theorem  are  always  "right"  in  an  ab¬ 
solute  sense,  even  in  a  simulated  environment.  Bayes’  Theorem, 
like  any  other  predictive  mechanism,  may  be  the  victim  of  ma,s- 
leading  information.  The  statistical  laws  ‘governing  the  simulated 
environment  usually  act  so  as  to.  ensure  that  the  information  fed 
into  the  predictive  mechanism  is  at  least  a  little  misleading , most 
of  the  time.  Under  these  circumstances  one  can  suspect  sleight  of 
hand  (or  worse)  if  .the  predictions  issuing  from  the  predictive 
mechanism  are  brighter"  than  they,  ought  to  be  according  to  Bayes’ 
Theorem.  :  • 

f  ,  • 

In  the  usual  sort  of  application  proposed  for  a  test  bed,  the 
predictive  mechanism  is  rated  by  comparing  its  predictions[  with 
the  "true"  state  of  affairs  over  a  ‘large  number  of  trials*.  Be¬ 
cause  on  any  given  trial  misleading  information  may  be  fed  into 
the  predictive  mechanism, ;some  way  must  be  found  of  estimating 
how  much  of  the  discrepancy  between  the  "true"  and  the  predicted 
state  of  affairs  is  to  be  attributed  to  the  predictive  mechanism,' 
and  how  much  to  misleading  information.  Misinformation  causes  a 
blurring  of  the  respective  merits  of  competitive  predictive  mech¬ 
anisms  being  compared  using  the  test  bed.  Moreover,  because  mis¬ 
information  is  itself  typically  present-  in  varying  amounts  from  . 
trial  to  trial,  a  larger  number  of  trials  is  usually  required  to 
average  out  the  statistical  fluctuations  than  would  be  the  case 
otherwise; 


Assuming  that  the  statistical  characteristics  of  pseudo¬ 
random  number  generators  employed  in  the  simulation  are  known. 
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These  effects  of  misinformation  can  be  eliminated  by  com¬ 
paring  the, output  of  the  predictive  mechanism  under  test  with  the 
output  of  Bayes'  Theorem  under  the  same  circumstances  on  a  trial 
by  trial  basis.  The  advantages  of  doing  this  are  apparent  from 
the  foregoing  discussion.  The  disadvantages  have  to  do  with  the 
co sc  and  difficulty  of  implementing  Bayes'  Theorem  in  this  way. 

As  more  is  learned  about  ways  to  compute  Bayes'  Theorem  economically 
these  disadvantages  will  diminish  (see,  for  example,  the  second 
paper  in  this  series) . 


c. 


ABSTRACT  STRUCTURE  OF  A  TEST  BED 


Like  the  portion  of  the  world  which  it  models,  a  test  bed 
can  be  structured  in  innumerable  ways.  The  next  chapter  pre¬ 
sents  design  concepts  for  the  test  bed  and,  therefore,  identifies 
its  components  according  to  their  physical  and  logical  structure. 
This  chapter  presents  some  theoretical  considerations  and,  there¬ 
fore,  deals  with  the  abstract  structure  of  the  test  bed.  The 
components  of  the  abstract  structure  are  identified  by  their  cor¬ 
respondence  to  the  entities ,  both  tangible  and  intangible ,  of  the 
real  world,  that  is,  by  the  entities  they  simulate.  Some  of  these 
entities  are  material  and  others  are  operations ,  activities ,  or 
behavior  displayed  in  the  real  world. 

Tables  1  through  3  list  these  components  by  the  name  of  their 
corresponding  entities.  For  clarity,  attributes  of  some  of  these 
entities  are  included.  These  tables  are  suggestive  rather  than 
comprehensive . 

There  are  a  large  number  of  real-world  interactions  among  the 
entities  represented  in  these  tables.  Development  of  a  concrete 
realization  of  a  test  bed  would  be  concerned  with  representing 
these  interactions  as  abstracted  interactions  among  test  bed  com¬ 
ponents.  Transpiration  of  component  interactions  simulate  the 
occurrence  of  real-world  events  and,  thence,  the  passage  of  time. 

Using  the  language  of  contingency  trees  developed  in  the 
first  paper  of  this  series,  all  possible  events  and  their  logical 
interrelationships  can  be  expressed  in  terms  of  a  contingency  tree 

Using  =»  slight  generalization"^  of  the  definition  of  a 


^The  generalization  is  as  follows..  Let  be  a  contingency  tree 
and  let  . (fn  be  classes  of  trees  such  that  (a)  for 


8 


\ 

contingency  plan,  the  "mission"  of  each  component  (i.e.,  the 
way  it  interacts  with  the  other  components  in  the  functioning 
of  the  test  bed)  can  then  be  expressed  as  a  complete  contingency 
plan  (corresponding  to  the  component  in  question)  for  t Con¬ 
versely,  rules  specifying  the  behavior  of  any  component  within  a 
te'st  bed  are  equivalent  to  a  complete  contingency  plan  for  that 
component. 

Partial  contingency  pians  may  be  assigned  to  test  bed  com¬ 
ponents  when  it  is  not  desired  to  specify  in  advance  their  func¬ 
tioning  under  all  possible  circumstances.  These  partial  contin- 

4 

gency  plans  may  be  refined  ad  lib  during  test  bed  operation.  In 
a  computerized  test  bed  this  process  may  be  expedited  through  the 
use  of  a  "freeze"  and  "defrost"  cycle,  as  explained  in  the  sub¬ 
sequent  chapter. 


all  and  for  all  ie{l,...,n}»  Let  be  the  class  of 

paths  in**/.  Then  for  any  given  ie{l, . . .  ,n} ,  is  a  class  of 
complete  contingency  plans  for  **/  with  respect  to 
^i+1' *  *  *  ,^n^iffi0l  f°r  all  (  <&1, . . . ,  ^)  e^x. .  .x(^n. 

Moreover ,  a  tree  ^9**#  is  a  complete  contingency  plan  for  **/  with 
respect  to  a  hyperclass  {  •  •  •  t  (Jn}  of  trees  satisfying  (a)  iff 

{^}  is  a  class  of  complete  contingency  plans  f  or  with  respect 

to  {  j  t  •  •  •  /  }  ♦ 


See  note  following  Definition  14  in  the  previous  paper  on  con¬ 
tingency  trees. 
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Table  1  -  MATERIAL  COMPONENTS  OF  A  TEST  BED 


•'  Platforms  {e.g.,  ships,  airplanes,  submarines,  decoys) 

Characteristics:  speed,  maneuverability,  capacity, 
other  limitations 

•  Equipment 

Number  and  distribution  of  each  item,  equipment  of 
each  kind-,  reliability,  vulnerability 

•  Resources  for  production  of  material 

•  Resources  for  repair 

•  Weapons 

•  Human  resources 

•  Resources  for  information  processing 
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Table  2  -  INFORMATIONAL  COMPONENTS  OF  A  TEST  BED 


•'  Sources 

Primary  Sources  (e.g.,  noise  of  engines) 

Secondary  Sources  (e.g.,  visibility  due  to  sun¬ 
light  or  moonlight;  active  sensors) 

Intelligence  Inputs 

•  Information  on  File  (e.g.,  ship  characteristics) 

•  Information  Processors ,  Command  and  Control  Centers 

Encryption  Devices  • 

Predictive  Mechanisms  (e.g.,  Bayes'  Theorem  Computer) 
Human  Resources 

•  Transmission  Channels 

•  Sensors 

Characteristics:  type  of  energy,  range  and  sensitiv¬ 
ity,  integration  time,  error  characteristics 


Detectable  by  passive  sensors  without  an  external  source  of 
energy. 

Detectable  by  passive  sensors  only  with  an  external  source  of 
energy,  whether  natural  or  artificial. 
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Table  3  -  TEST  BED  COMPONENTS  REFLECTING  BEHAVIOR  OF  NATURE 


•'  Material  constraints  (e.g.,  weather,  geography:  random 
failure  of  equipment) 

•  Informational  constraints  (e.g.,  false  contacts,  noise 
in  transmission  channels) 

•  Source  of  random  numbers 
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D. 


SUBMARINE  MISSIONS  AS  CONTINGENCY  PLANS  AND  THE  PROBLEM 
OF  COMPLETENESS 


It  follows  from  the  remarks  above  that  the  class  of  all 
possible  submarine  missions  is  co-extensive  with  the  class  of 
all  possible  complete  contingency  plans  for  submarines .  This 
class  if  limited  initially  by  the  specification  of  the  contin¬ 
gency  tree  and  is  further  limited  by  whatever  constraints 
may  be  imposed  on  the  classes  of  contingency  plans  available  to 
other  test  bed  components.  If  is  finite,  for  example,  the 
number  of  expressible  or  effective  submarine  missions  must  like¬ 
wise  be  finite. 

Every  actual  submarine  mission  may  be  intended  to  fulfill 
some  combination  of  commercial,  economic,  military,  political, 
research,  and  training  objectives.  Lists  of  ways'  in  which  sub¬ 
marines  could  be  used  to  contribute  to  these  and  perhaps  other 
objectives  can  be  made  up  by  interviewing  and  consulting  with 
experts  and  by  literature  search.  Possible  or  conceivable  sub¬ 
marine  missions,  suggested  in  unclassified  sources  or  by  the 
present  author's  imagination,  include  the  following: 

•  Transporting  goods  or  personnel  in  towed  underwater 
"barges " . 

•  Locating  and  developing  or  exploiting  underwater 
animal  (fish)  and  oil  or  mineral  resources. 

•  Supporting  sea-floor  farming. 

•  Utilizing  and  supporting  underwater  storage  depots 
or  staging  areas. 

•  Assisting  in  search  and  rescue  operations  (e.g.,from 
other  submarines). 

•  Mapping  ocean-floor  topography,  geology  (including 
sedimentation)  and  temperatures. 

•  Charting  ocean  temperatures,  currents,  and  variation 
in  salinity. 
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•  Clandestinely  supplying  and  supporting  guerilla  or 
insurgent  forces  (or  de  jure  governments,  for  that 

’matter) . 

•  Leaving  off  or  picking  up  intelligence  agents. 

•  Probing  enemy  ASW  defenses. 

•  Locating,  shadowing,  or  harassing  potentially  hostile 
naval  forces. 

•  Screening  friendly  naval  forces. 

•  Protecting  harbors. 

•  Laying  mines. 

•  Detecting  and  destroying  enemy  submarines. 

•  Providing  a  missile  "deterrent"  to  war. 

•  Monitoring  shipping  and  electronic  communications. 

•-  Obtaining  "signatures"  of  potentially  hostile  enemy 
ship  operations,  especially  submarines. 

•  Monitoring  naval  operations  and  exercises. 

•  Providing  fire  support  for  small-scale  amphibious  land¬ 
ings  . 

•  Providing  a  temporary  hiding  place  for  (what  is  later 

to  become)  a  command-post  staff,  in  the  event  of  all-out 
war. 

•  Providing  miscellaneous  provocations  in  a  cold-war  con¬ 
text.  Some  of  the  military  missions  might  involve  a 
"show  of  force".  Even  some  of  the  ostensibly  peaceful 
missions  may  have  significant  military  and  political 
overtones . 

.•  Conceivable  expanded  submarine  capability,  such  as 
flying,  boring  underground,  or  traveling  overland. 


It  would  be  desirable  to  expand  the  list  of  missions  given 
in  the  preceding  paragraph.  In  fact,  it  would  be  desirable  to 
attempt  a  list  of  "all  possible"  submarine  missions,  together 
with  an  explication  of  "all  possible".  The  discussion  accompanying 
such  an  explication  should  make  clear  the  basis  of  information  used 
in  deciding  whether  something  is  a  possible  submarine  mission  or 
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not,  and  should  give  some  indication  of  (a)  how  the  class  of 
possible  missions  would  change  if  the  information  base  were  to 
change,  (b)  how  likely  the  information  base  is  to  change,  and 
(c)  in  what  ways  the  information  base  is  likely  to  change. 

The  purpose  of  such  a  list  is  two-fold.  First,  it  would 
provide  a  check  on  the  adequacy  of  a  proposed  contingency  tree 
for  the  submarine  missions.  Second,  judgments  of  the  degree 
and  effects  of  variability  of  the  information  base  mentioned 
above  would  be  explicitly  realized  and  available  as  qualifications 
on  the  conclusions  developed  from  test  bed  runs.  The  problem  of 
information  base  variability  and  completeness  has,  in  the  author's 
opinion,  received  too  little  attention,  particularly  with  respect 
to  its  critical  impact  on  the  validity  of  models  and  applications 
of  models.  A  subsequent  paper  in  this  series  considers  this  prob¬ 
lem  more  completely. 

Even  a  partial  list  of  submarine  missions  permits  a  {cor¬ 
respondingly  partial)  check  on  the  adequacy  of  a  proposed  contin¬ 
gency  tree  If,  for  instance,  is  not  sufficiently  inclusive 

to  permit  a  contingency  plan  c^rt^to  be  formulated  corresponding 
to  some  possible  submarine  mission,  then  either  must  be  amended 
or  a  decision  must  be  reached  to  accept  the  error  inherent  in  not 
amending  An  error  analysis  is  desirable  in  the  latter  case. 

The  adequacy  of  for  formulating  complete  contingency  plans 
corresponding  to  other  test  bed  components  must  be  verified  simi¬ 
larly.  In  fact,  the  class  ^  of  paths  in  tjtf.  must  adequately  mir¬ 
ror  all  possible  sequences  of  events  in  the  real  world.  Here, 
again,  there  are  problems  with  defining  what  "adequate"  means. 
Conceptually  it  is  possible  to  be  concerned  directly  with  the 
adequacy  of  and  only  indirectly  with  the  adequacy  of  the  list 
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of  submarine  missions;  but  computationally  it  is  desirable  that 
be  no  larger  than  is  necessary  for  realism.  Inclusion  of 
only  submarine  missions  specifically  considered  to  be  possible 
frequently  "prunes"  the  contingency  tree  and,  therefore, 
eliminates  undesirable  members  of 


E. 


THE  TEST  BED  AS  A  GENERAL  SUM  GAME 


By  assigning  sets  of  utilities  to  all  ,  the  test 

bed  takes  the  form  of  an  n-person  general-sum  game.  Because 
the  test  bed  typically  utilizes  a  chance  device  (e.g.,  a 
pseudo-random  number  generator)  with  known  probability  char¬ 
acteristics,  the  expected  utility  for  each  set  of  utilities 
may  likewise  be  thought  of  as  having  known  probability  char¬ 
acteristics.  But  that  is  only  true  after  each  player  has 
chosen  his  strategy  (i.e.,  his  complete  contingency  plan  or, 
more  generally,  the  probability  with  which  he  will  use  each 
available  complete  contingency  plan) .  There  is  no  generally 
accepted  definition  of  a  "solution"  to  general-sum  games  (see 
the  second  paper  in  this  series) ,  and  observed  behavior  may 
vary  widely  from  player  to  player.  Therefore,  although  gen¬ 
eral-sum  game  theory  provides  boundaries  and  general  guidelines 
for  choosing  one's  own  strategies,  it  usually  does  not  give  a 
specific  "mix"  of  strategies  (except  under  controversial  ad¬ 
ditional  assumptions)  either  for  oneself  or  one's  opponents,  and 
a  fortiori  does  not  yield  a  priori  probabilities  of  an  opponent 
playing  his  various  available  pure  strategies.  General-sum  game 
theory  can,  however,  suggest  ranges  of  a  priori  probabilities  that 
should  be  considered,  and  sensitivity  analysis  may  be  conducted 
within  these  ranges. 


As  pointed  out  in  earlier  papers,  there  are  problems  of  as¬ 
signing  utilities  to  paths  in  contingency  trees.  In  the  present 
connection  two  distinct  but  related  problems  arise:  assigning 
utilities  from  one's  own  point  of  view,  and  attributing  utilities 
to  other  players.  The  uncertainties  associated  with  the  latter 
process  are  an  additional  source  of  theoretical  and  practical  dif¬ 
ficulties. 
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F,  PROBLEMS  IN  ESTIMATING  PROBABILITIES 

—  — - * -  |  * 

i  i 

* 

A  priori  probabilities  determined  from  game  theory  or 

,  i 

otherwise  may  be  made  available  to  a  Bayesian  predictor  within 
the  test  bed.  Other  predictive  mechanisms.' may  be  used  alter-' 
natively.  A  measure  of  information  loss,  applicable  in  situations 
where  the  Bayesian  information  processor  may' be  used  normatively, 

•  :  •  '5  1  f 

is  suggested  m  a  previous  paper  .  This  measure  consists  of  the 
expected  search  time,  under  the  conditions  set  forth  there,  using 
an  optimal  Search  strategy  based  on  the  alternative  predictive 
mechanism  and  assuming  an  actual  distribution  based  on  the  re¬ 
sults  of  the.  Bayesian  predictor,  minus  the  similarly  defined  ex¬ 
pected  search  tim.e  using  an  optimal  search  strategy  based  on  the 
results  of  the  Bayesian  predictor  and  assuming  an  actual  distri¬ 
bution  which  is  also  based  on  the  results  of  the  Bayesian  pre¬ 
dictor.  When  only  ranges  of  a  priori  probabilities  ,are.  available 
to  the  Bayesian  predictor,  a  corresponding  range  of  possible  in¬ 
formation  loss  may  be  computed  similarly. 

.  „  '  .  ■  I 

Generally  speaking,  estimating . probabilities  of  the  form 
P[D|H]  presents  a  less  severe  problem  in  ASW  contexts  than  estimating 
the  a  priori  probabilities.  This  is  because  analytic  models  for 

f 

sensor  performance  under  various  environmental  conditions  are  usu¬ 
ally  available,  together  with  data  suitable  for  confirming  or  re¬ 
jecting  the  analytic  m'odels  and  for  determining  the  parameter  set¬ 
tings  within  the  models.  The  present  author  is  far  from  satisfied 

I 

with  presently  available  models  and  their  validations,  based  on 
what  he  knows  about  them.  So  these  remarks  are  not  intended  to 
suggest  that  present  knowledge  in  these  areas  is  adequate;  quite 
the  contrary.  But  at  least  the  problems  are  recognized  and  fairly, 
well  defined,  and  promise  to  be  of  continuing  interest  to  the  Navy 


See  the  topic  "A  Measure  of  Information  for  Search  Theory"  in 
the  second  paper  in  this  series. 
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for  at'  least  as  long  as  choices  between  rival  sensor  systems 
must  be  made  and  justified.  Provided  that  conditions  where 
a  very  slight  change  in  environmental  conditions  -  one  which 
it  is  impractical  to  measure  under  ordinary  conditions  -  can 
■  cavlse  a  drastic  change  in  sensor  capabilities^,  provided  that 
these  cqnditions  do  not  occur  too  frequently,  and  provided  that 
careful  research  in  this  area  is  continued,  it  should  be  only  a 

•  i 

matter  of  time  before  acceptable  models  for  determining  P[DlH] 
are  available  for  most  sensors  of  interest.  'Predictive  mechanisms 
like  Bayesian  processors,  which  require  P[DIH]  as  inputs  may  there 
fore  be  constructed  so  as  to  accept  present  values  of  P[D|H],  but 
so  as  to  be .adaptable  to  new  values  of  P[D|H]  as  they  become  avail 
able. 


6  Cf. [Tolstoy  and  Clay,  1966,  P.  271] 
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III.  DESIGN  CONCEPTS 


A.  INTRODUCTION 


This  chapter  sets  forth  some  of  the  design  principles 
and  objectives  for  a  test  bed  of  ASW  predictive  mechanisms. 
Test  bed  operation  is  conceived  as  having  the  following 
steps.  First,  an  analyst  easily  formulates  and  inputs  a 
scenario.  Second,  the  analyst  inputs  a  submarine  predic¬ 
tion  method  or  ASW  threat  evaluation  system,  Bayesian  or 
otherwise,  associated  with  one  side  of  the  ASW  engagement. 

The  test  bed  then  serves  its  first  function  by  simulating 
the  actions  of  all  forces  involved  m  the  scenario.  At 
specified  simular  times,  it  then  outputs  the  system’s  pre¬ 
diction  and  associated  measures  of  effectiveness.  The  pre¬ 
dictive  threat  evaluation  system  may  or  may  not  (as  specifie 
by  the  analyst)  influence  the  actions  of  the  forces  associated 
with  its  side  during  the  simulation. 

The  principal  advantages  of  using  the  test  bed  rather 
than  sea-going  exercises  are  obvious:  (a)  savings  in  cost 
and  time,  (b)  complete  experimental  control  over  the  experi¬ 
ment,  and  (c)  the  impossibility  or  impracticability  of  emu¬ 
lating  many  scenarios  at  sea. 
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B. 


DESIGN  OBJECTIVES 


For  the  test  bed  to  be  a  practical  and  effective  tool 
the  following  objectives  for  its  design  seem  reasonable: 


(1)  The  test  bed  program  should  be  limited  to  a  reason¬ 
able  amount  of  core  storage. 

(2)  The  test  bed  program  should  require  a  minimal 
amount  of  computation  time.  In  many  cases,  the 
analyst  will  want  to  perform  many  trials  with  the 
same  scenario  to  obtain  some  statistical  signifi¬ 
cance  for  his  findings.  This  will  be  impractical 
if  the  test  bed  program  runs  slowly. 

(3)  The  test  bed  should  be  capable  of  simulating 
scenarios  of  different  scales,  from  a  few  ve¬ 
hicles  near  datum  to  all  ASW  forces  in  a  large 
portion  of  an  ocean. 

(4)  The  test  bed  should  be  designed  to  minimize  the 
analyst :s  effort  and  time  in  preparing  scenario 
inputs . 

(5)  The  test  bed  should  enable  the  scenario  to  be  a 
truly  two -sided  interaction  between  the  opposing 
forces;  as  opposed  to  fixing  the  behavior  of  one 
side  (usually  the  enemy's)  while  allowing  flexi¬ 
bility  of  response  on  the  other. 

(6)  The  test  bed  should  be  capable  of  accepting  scen¬ 
arios  which  include  any  type  and  number  of  plat¬ 
forms,  sensors,  and  weapons. 

(7)  The  test  bed  should  be  capable  of  simulating  action 
of  a  platform  appropriate  to  any  possible  mission. 
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c. 


GENERALIZED  TREATMENT  OF  VEHICLES 


1 .  Module  Concept 

To  achieve  the  flexibility  and  economy  of  space  desired 
in  the  test  bed  program,  all  ships,  aircraft,  submarines, 
etc.  (in  fact,  all  vehicles)  will  be  represented  in  the  pro¬ 
gram  as  modules .  In  this  way  separate  subroutines  and  con¬ 
siderations  for  each  type  of  vehicle  can  be  avoided.  By 
sufficiently  generalizing  the  descriptive  parameters  of  such 
modules,  the  same  set  of  subroutines  can  simulate  the  actions 
of  all  vehicles. 

Further,  the  notion  of  a  module  is  extended  to  cover 
other  elements  of  the  ASW  problem  such  as  buoys,  torpedoes, 
command  and  control  centers,  and,  in  fact,  every  possible 
unit  of  action,  to  achieve  the  maximum  degree  of  program 
savings  and  flexibility  of  use. 

A  module  then  is  any  physical  element  of  the  scenario 
which  is  capable  of  processing  information  and/or  performing 
an  action  of  interest  to  the  analyst  for  a  problem  at  hand. 

Another  generalization  is  fragmentation  and  coalescence 
of  modules.  To  accommodate  the  generalization  of  such  dis¬ 
parate  items  as  ships,  torpedoes,  helicopters,  buoys,  etc., 
it  will  be  necessary  to  have  a  scheme  which  allows  a  module 
to  carry,  launch,  control,  and  recover  other  modules. 

2 .  Module  Vector 

To  describe  and  specify  a  particular  module,  a  list  of 
the  descriptive  parameters  for  all  modules  will  be  required. 
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This  vector  can  be  further  subdivided  into  two  parts,  as 
follows:  . 

*  Fixed  module  vector  containing  parameters  whose 
values  are  fixed  or  invariant  during  a  run. 

'  •  Variable  module  vector  containing  parameters 

whose  values  may  vary  during  a  run. 

Tables  4  and  5  suggest  sets  of  parameters  for  the  above 
two  types  of  vectors,  respectively.  As  the  need  arises, 
these  can  be  appended  to  incorporate  other  parameters . 

For  the  purpose  of  fragmentation  and  coalescence  of 
modules,  the  fixed  module  vector  contains  the  identification 
of  its  carrier  module,  if  any.  The  carrier  module  is  the 
"mother  ship"  of  a  helicopter,  torpedo,  or  buoy,  as  examples, 
as  well  as  the  literal  aircraft  carrier  of  a  VS  aircraft  or 
the  shore  base  of  V"  aircraft. 

The  variable  module  vector  contains  two  items  necessary 
to  specify  the  carrier/carried  relationship.  Item  number  5 
is  the  "aboard  vs.  launched"  indicator.  This  specifies  the 
status  of  the  module  being  described  in  relation  to  its  car¬ 
rier  .module  (if  any) .  As  long  as  this  module  is  still 
"aboard"  its  carrier,  it  will  be  ignored  by  the  program. 

Only  if  it  is  launched  and  alive  will  the  module  be  processed 
for. its  possible  actions  and  effects.  The  other  item,  num¬ 
ber  6,  specifies  the  number  of  carried  modules  by  type  remain¬ 
ing  aboard  this  module. 

Finally,  to  exhaust  the  list  of  fundamental  variables 
treated  by  the  test  bed  simulation  there  is  a  requirement  for 
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parameters  representing  certain  relations  among  modules,  the 
simplest  example  being  the  true  range  between  modules.  The 
tables  which  specify  the  values  between  all  possible  pairs 
of  modules  are  called  inter-module  vectors  whose  variables 
are  listed  in  Table  6. 
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Table  4  -  FIXED  MODULE  VECTOR  COMPONENTS 


'Type  of  Module 
Identification  of  Module 
Mission  of  Module 
Turn  Radii 

Speed  vs.  Noise  Curves  (Also  gives  maximum  speed) 

Carrier  Module  (Identification  of  module  carrying  this  one) 

Type  Modules  Carried 

Communications  From/To  Modules 

Type  Detectors 

Nationality  of  Module 
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Table  5  -  VARIABLE  MODULE  VECTOR  COMPONENTS 


1.  Position 

2 .  Heading 

3.  Altitude 

4.  Self  Noise 

5.  Radiated  Noise 

6.  "Aboard  vs.  Launched"  Indicator 

7.  Number  of  Carried  Modules  Remaining  by  Type 

8.  Turn  Radius;  R 

9.  Turn  Direction;  TD 

10.  Speed;  S 

11.  Detector  Mode,  by  Type  (active,  passive) 

12.  Detector  Status,  by  Type  (operational,  in  repair,  etc 

13.  Noise  State 

14.  Communication  Mode,  by  Type 

15.  Communication  Status,  by  Type 

16 .  Dead/Alive  Status 
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Table  6 


INTER-MODULE  VECTORS 


True  Range  Between  Modules 

True  Relative  Bearing  Between  Modules 

Noise  Radiated  From  One  Module  to  Another  (passive-one  way) 
Noise  Reflected  Off  One  Module  by  Another  (active-two  way) 
Detection  Signal  Recognized  on  One  Module  By  Another 
Classification  Of  One  Module  By  Another 
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D.  TEST  BED  RULES 

1.  General  Concept 

Test  bed  rules  are  the  total  set  of  logical  and  numeric 
equations  or  inequality  relations  which  determine  the  changes 
of  state  of  the  modules  throughout  the  simulation  process. 

They  are  the  mechanisms  which  control  the  simulation.  At  the 
time  of  actual  operation  of  the  test  bed,  the  test  bed  rules 
virtually  constitute  the  computer  program  logic  itself,  with 
the  exception  of  that  part  of  the  program  which  might  be 
termed  the  master  control  program.  This  latter  is  envisaged 
as  merely  a  skeleton  frame  work  of  a  small  number  of  instruc¬ 
tions  which  serves  to  control  the  timing  and  use  of  the  test 
bed  rules.  The  master  control  program  is  the  invariant  struc¬ 
ture  needed  in  any  particular  test  bed  simulation. 

The  test  bed  rules  are  divided  into  four  categories  to 
simplify  and  standardize  the  methods  required  to  implement 
a  simulation.  The  four  categories  have  been  labelled: 

•  Set-Up  Rules 

•  Perception  Rules 

•  Action  Rules 

•  Simulation  Rules 

The  set-up  rul^s  take  all  of  the  simulation  run  inputs 
prepared  by  the  analyst  and  prepare,  as  input  to  the  simula¬ 
tion,  the  values  of  all  time-invariant  parameters  as  well  as 
the  initial  values  of  certain  dependent  variables.  Essentially, 
these  rules  describe  the  initial  set-up  of  the  program. 
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The  perception  rules  may  be  defined  as  a  set  of  inequal¬ 
ity  statements  about  any  type  of  variable  which  is  used  to 
determine  the  perceived  values  of  any  variable  of  interest. 

A  perceived  value  is  based  on  the  true  value  of  a  variable 
determined  by  sensor  system  and  contains  an  error  represen¬ 
tative  of  the  sensor  system  performance  and  human  interpre¬ 
tation.  Certain  perception  rules  themselves  will  be  regarded 
as  the  main  variable  of  experimental  interest  in  the  use  of 
the  test  bed.  These  rules  are  used  to  describe  (or  simulate) 
all  sensor  actions  in  the  test  bed  and  to  describe  all  threat 
evaluation  functions  (i.e.,  predictions  concerning  the  enemy). 

The  action  rules  are  sets  of  inequality  statements  about 
perceived  values  used  to  determine  the  settings  of  control 
variables.  The  action  rules  therefore  correspond  to  the 
action  selection  functions  of  the  various  modules.  The  action 
rules  for  a  module  are  by  definition,  a  complete  specification 
of  the  mission  of  the  module,  since  any  action  of  any  char¬ 
acter  for  each  module  must  be  pre-specif ied  by  action  rules . 
Differences  among  missions  correspond  to  differences  among 
action  rules  fcr  similar  modules. 

The  simulation  rules  are  inequality  statements  about 
true  values  of  any  kind  of  variable  used  to  determine  the 
settings  of  all  dependent  variables.  These  rules  are,  for 
the  most  part,  laws  of  physics  used  to  drive  the  simulation 
forward  through  time.  The  label  "simulation  rules"  is  used 
because  these  rules  control  simulation  of  physical  reality. 

The  following  paragraphs  elaborate  on  the  use  of  each 
kind  of  rule. 
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2 .  Perception  Rules 

Under  this  heading  come  rules  for  setting  the  values 
of  variables  needed  by  a  module  to'  implement  its  action  rules , 
but  which  are  not  directly  accessible  by  the  module  (such  as 
almost  all  variables  pertaining  to  other' modules) ;  these  values 
must  therefore'  be  either: 

i 

e  Communicated  to  the  module  from  other  modules, 

1  .  i 

o  Perceived  by  the  module  via  sensors , 

•  Deduced  by  the  module  from,  past  perceptions ,  or  , 

©  Assumed  by  the  module. , 

I 

The  values  of  all  such  variables,. when  set  or  determined,  are 
called  perceived  values  to  distinguish  them  from  the  true 
values  of  these  variables . 

The  perception  rules  for  a  particular  module,  when  exe¬ 
cuted,  produce  a  "picture"  which' represents  surrounding  reality 
as  perceived  by  the  module.  If  a  module  represents  an  ASW 
command  and  control  center,  then  the  perceived  picture  and 
its  method  of  production  is  the  principal  item  of  interest 
for  which  the  test  bed  is  being  exercised. 

If-'  the  module  represents  a  ship  or'  aircraft,  then  the 
picture  may  include  the  location,'  classification,  speed,  and 
heading  of  all  modules  of  interest  to,  and  perceived  by 
that  module.  The  above  four  variables  wil_]  ,  generally  speak¬ 
ing,  be  the  variables  most  generally  required  by  the  action 
rules. 
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3.  Action  Rules:  Canonical  Representation  of  a  Mission 

I 

Each  module  in  a  simulation  must  be  given  a  mission.  A 
mission  is  defined  for  the  test  bed  by  a  mission  profile 
which  is  equivalent  to  a  set  of  action  rules  for  the  module. 
This  mission  profile  must  specify  actions  for  all  situations 
which  may  be  encountered  by  the  module. 

I 

«  i 

Specification  of  these  rules  is  not  quite  as  complex  a 
task' for  the  analyst  as  it  appears  at  first,  for  the  analyst 
is  aware  of  the  whole  scenario  which  he  intends  to  simulate 
and  hence  need  only  specify  actions  for  the  possible  situa¬ 
tions  he  knows  a  module  will  encounter.'1'  Secondly,  some 
modules  such  as  weapons,  buoys,  etc.  will  not  need  mission 
profiles  tailored  to  the  scenario.  Once  these  are  formulated 
for  any  one  scenario  they  may  be  used  for  many  subsequent 
scenarios.  This  leads  to  the  idea  of  gradual  accumulation 
of  a  mission-profiles  library  from  which  the  analyst  can 
select  ready-made  mission  profiles. 

To  facilitate  the  computer  economies  referred  to  above, 
the  action  rules  will  be  input  to  the  test  bed  in  the  form 
outlined  below. 

For  each  module  a  set  of  single  valued  functions,  X(I) 
fpr  1=1, 2,..., M  are  specified.  The  X(I)  are  to  be  functions 


iThis  is  not  to  say,  however,  the  evaluation  of  test  bed 
results  based  on  using  scenarios  which  deal  with  part  of  the 
contingency  tree  representing  missions  of  a  module  is  re¬ 
lieved  of  the  problems  of  completeness  identified  in  Chapter  II 
of  this  paper. 
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of  any  variables  perceivable  by  the  given  modules.  Next  a 
set  of  comparison  values  or  thresholds,  L ( I )  for  1=1, 2,..., M, 
are  defined.  These  functions  and  values  are  then  structured 
so  that  a  three-fold  branch  occurs  accordingly  as  X(I)  is 
less  than,  equal  to,  or  greater  than  L(I)  for  each  I.  Each 
branch  either  identifies  another  rule  or  a  setting  function. 
The  set  of  all  rules  forms  one  or  more  trees  whose  terminuses 
("leaves")  are  identifications  of  setting  functions.  The 
setting  functions  are  algorithms  which  alter  one  or  more  con¬ 
trol  variables  for  the  module,  and,  hence,  affect  subsequent 
actions  of  the  module. 

The  canonical  form  forces  the  analyst  to  consider  the 
possibility  of  all  possible  outcomes  relative  to  the  questions 
he  asks  (the  set  of  comparisons  indexed  by  I)  about  the  cur¬ 
rent  situation  facing  the  module.  As  a  result,  the  mission 
is  completely  defined  to  that  extent.  The  test  bed  is  as¬ 
sured  therefore  that  it  always  knows  what  to  do  with  and  for 
every  module  at  any  time.  Whether  the  mission  is  realistic 
or  well-defined  for  the  purposes  at  hand  is  another  matter. 

4 .  Simulation  Rules 


As  defined  above  the  simulation  rules  serve  to  determine 
the  physical  effects  of  all  variables  at  each  time  sample.2 
These  rules  represent  the  domain  of  physics  (or  psycho-physics 
in  the  case  of  man-machine  systems  such  as  most  sensor  systems). 
There  are  several  categories  of  simulation  rules  which  are 
evaluated  for  each  time  break  T.  The  rules  fall  in  these  cate¬ 
gories  respectively: 

(1)  Effect  any  launches  of  carried  modules  as  "ordered" 
by  carrier  modules  at  time  T. 

2Assuming  a  time  step  simulation. 
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(2)  Effect  movement  of  all  modules;  that  is,  given  the 
true  values  of  all  variables  at  time  T,  determine 
“the  new  positions  and  headings  for  time  T+AT. 

(3)  Determine  the  outputs  of  all  detection  systems  for 
T+AT.  These  outputs  will  include  possible  errors 
concerning  real  targets  as  well  as  outputs  produced 
by  false  targets  of  various  types. 

(4)  Determine  kills  or  damage  to  modules  due  to  impacts 
or  explosions  of  other  modules  over  the  interval 
T+AT . 

(5)  Effect  recovery  of  certain  modules  by  carrier  mod¬ 
ules  (e.g.,  as  in  the  case  of  helicopters  returning 
to  destroyers  or  aircraft  to  carriers  or  base,  etc.) 
at  time  T+AT. 

(6)  Effect  transmission  (when  legal)  of  contact  data 
and  action  orders  between  modules  as  these  are  re¬ 
quested  by  the  modules . 


For  certain  types  of  analysis,  simulation  of  details  may 
not  be  necessary.  In  these  cases,  simple  artificial  weapon 
systems  may  be  formulated  for  the  sake  of  speed  and  ease  of 
the  simulation. 


On  the  other  hand  the  test  bed  should  be  capable  of  sim¬ 
ulating  detailed  effects,  interactions,  and  operations  of 
sensors  and  weapon  systems  for  those  applications  wherein  the 
availability  of  detailed  data  and  the  objectives  for  applying 
the  test  bed  warrant'  such  detail. 
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E.  OTHER  STRUCTURAL  CONCEPTS 

1.  False  Contacts 

False  contacts  may  also  be  included  in  the  simulation 
scheme  by  means  of  the  module  concept.  Each  type  of  false 
contact  to  be  included  can  be  formulated  as  a  type  of  module 
with  its  own  set  of  action,  perception  rules,  etc.  Since 
false  contact  as  defined  here  is  any  contact  not  on  an  enemy 
module  or  on  one  of  the  known  friendly  modules ,  the  list  of 
possible  false  contacts  that  could  be  simulated  include: 

•  Merchant  shipping 

c  Whales 

•  Biologies 

e  Underwater  sea  mounts 

5*  Wrecked  ships 

•  Knuckles 

In  the  case  of  false  contacts  such  as  knuckles  and  some 
type  of  biologies  some  provision  must  be  made  for  their  "birth" 
and  "death". 

The  birth  of  such  modules  cannot  be  handled  so  simply 
in  terms  of  the  concepts  discussed  so  far.  It  requires  a 
subprogram  which  will  determine  which  type  of  false  contact 
module  will  be  born  at  what  times  and  at  which  locations, 
based  on  inputs  supplied  by  the  analyst. 

2.  Contact  Table 

Each  module  which  possesses  a  sensor  or  which  may  be  in 
communication  with  sensor-processing  modules  will  have  a 
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standard  format  contact  table.  This  table  will  describe  eacn 
individual  contact  of  current  interest  to  the  module.  The 
table  items  will  include: 

•  Originating  Module 

'  •  Originating  Sensor  Type 

•  Originating  Module  Location 

•  Originating  Module  Control  Parameters  (heading, 

speed,  etc.) 

•  Originating  Module  Classification  Data 

•  Latest  Time  of  Contact 

•  Contact  Range 

•  Contact  Bearing 

9  Contact-Signal-Presence-Only  Markers  (for  sensors 
not  giving  range  or  bearing  data) 

•  This  Module's  Classification  Data 

The  purpose  of  the  table  is  to  provide  a  standard  format 
for  storage  of  contact  data  for  present  and  future  classifica¬ 
tion  and  correlation  and  for  localization  of  the  contact  by 
this  module  or  other  modules  to  which  this  data  may  be  trans¬ 
mitted. 

3.  Communication  Between  Modules 


Rules  governing  communication  between  modules  will  be 
used  to  simulate,  when  desirable,  the  effects  of  communication 
constraints  on  transmission  of  contact  (i.e.,  threat  evaluation) 
data  and  transmission  of  action  orders  by  higher  to  lower 
echelon  modules  (i.e.,  action  selection  data).  For  the  pre¬ 
sent  time  the  possibility  of  communications  generating  con¬ 
tact  data  for  the  opposing  side  will  not  be  considered.  Further, 
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it  will  be  assumed  that  the  mode  of  communication  used  is 
secure  from  signal  disruption  by  the  enemy.  To  co'-a-ol  simu¬ 
lation  of  transmissions,  the  action  rules  are  used  to  specify 
the  initiation  of  any  communications  by  a  module,  either  to 

send  or  to  receive  information. 

«• 

Transmission  of  contact  data  will  be  simulated  by  trans¬ 
ferring  appropriate  parts  of  the  contact  table  of  one  module 
into  the  contact  table  of  the  receiving  module.  If  an  action 
order  is  to  be  transmitted,  the  ordering  module  will  simply 
transmit  the  necessary  setting  function  data  to  the  receiving 
module.  The  action  rules  of  the  latter  module  must  therefore 
contain  a  query  as  to  the  receipt  of  such  orders  and  there¬ 
upon  process  setting  function  data  to  implement  the  orders . 

4.  Unique  Actions 


It  is  not  to  be  supposed  that  all  actions  of  interest 
must  be  reducible  to  rules  in  advance  of  the  simulation. 
There  will  be  provisions  for  the  analyst  to  preprogram  opera¬ 
tion  of  the  test  bed  so  as  to  cause  suspension  of  simulation 
operation  under  certain  conditions.  The  analyst  will  then 
be  presented  with  the  option  of  making  manual  decisions  and 
then  resuming  simulation  operation  with  the  results  of  the 
decision  inserted  into  the  logic  flow. 

To  achieve  this  capability  the  test  bed  will  incorporate 
the  computer  functions  called  "freeze"  and  "defrost".  If  the 
test  bed  is  instructed  to  freeze,  it  will  stop  the  simulation 
process  and  display  the  current  situation  as  perceived  by  the 
module  whose  action  rules  instigated  the  freeze.  It  will  then 
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copy  the  pertinent  sections  of  core  memory,  including  the 
program,  o.nto  auxiliary  memory  and  terminate  computer 
operations . 

The  analyst  then  prepares  his  decisions  concerning  the 
situation  (as  requested  by  the  display)  and  inputs  them  to 
a  separate  program  called  defrost.  The  defrost  program  will 
read  in  the  original  copy  of  the  program  and  storage  contents 
(which  preserve  the  undecided  situation)  along  with  the  re¬ 
quested  action  decisions.  After  inserting  the  decisions  as 
action  selections  into  the  program,  control  is  returned  to 
the  test  bed  program  which  then  resumes  its  processing  at  the 
point  of  the  original  interruption. 

By  preserving  auxiliary  memory  media,  the  frozen  situ¬ 
ation  can  be  restarted  any  number  of  times  to  explore  the 
effect  of  different  action  options,  should  this  be  desired. 

5.  Data  Recording 

This  function  of  the  test  bed  program  is  to  record  data 
onto  tape  during  the  simulation  which  may  in  turn  serve  as 
inputs  to  the  computation  of  measures  of  effectiveness,  dis¬ 
plays,  simple  frequency  counts,  and  other  forms  of  summari¬ 
zation  of  a  run  or  group  of  runs. 

The  analyst  will  control  the  recording  of  data  by  speci¬ 
fying  variables  and  modules  of  interest  and  the  time  of  re¬ 
cording.  He  may  specify  that  data  be  recorded  at  fixed  simu¬ 
lation  times,  or  periodically  with  specified  time  periods, 
or  conditionally  upon  a  given  event  occurrence. 
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